REPRESENTATIONS WHOSE MINIMAL REDUCTION 
HAS A TORIC IDENTITY COMPONENT 



CLAUDIO GORODSKI AND ALEXANDER LYTCHAK 

Abstract. We classify irreducible representations of connected compact Lie groups whose 
orbit space is isometric to the orbit space of a representation of a finite extension of (positive 
dimensional) toric group. They turn out to be exactly the non-polar irreducible representa- 
tions preserving an isoparametric submanifold and acting with cohomogeneity one on it. 



1. Introduction 

A representation p : G — > 0(V) of a compact Lie group on an Euclidean space V is 
called polar if there exists a subspace E, called a section, that meets all G-orbits and meets 
them always orthogonally jPT87| IBCO 03J. In this case, the normalizer iV(£) acts on E 
with kernel equal to the centralizer Z(E), the quotient group W = iV(E)/Z(E) is finite, 
and the inclusion E — > V induces an isometry between orbit spaces E/W = V/G with 
the quotient metrics. Conversely, assume p : G — > O(V) is a representation whose orbit 
space X — V/G can also be given as the orbit space of a representation of a finite group. 
On one hand, the set of regular points X reg is a flat Riemannian manifold. On the other 
hand, due to O'Neill's formula, the horizontal distribution of the Riemannian submersion 
V reg — > X reg is integrable. It follows that p is polar [Ale06[ IHLO06] . Thus we have the 
following characterization: a representation is polar if and only if its orbit space is isometric 
to the orbit space of a representation of a finite group. 

This paper follows the program introduced in |GLj . namely, to hierarchize the represen- 
tations of compact Lie groups in terms of the complexity of their orbit spaces, viewed as 
metric spaces. Dadok [Dad85j has classified polar representations of connected groups and 
showed that they are orbit-equivalent to the isotropy representations of symmetric spaces. 
Herein we enlarge the class of polar representations by replacing "finite group" by "finite 
extension of toric group" in the above characterization. Namely, we consider a class of rep- 
resentations defined by the condition that the orbit space is isometric to the orbit space of 
a representation of a finite extension of toric group, and we classify such representations in 
the irreducible case and for connected groups. 

It turns out that such representations have a remarkable connection with isoparametric 
submanifolds and polar representations. And, rather surprisingly, only few families of non- 
polar representations can occur. Our result can be stated as follows. 
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Theorem 1.1. Let p : G — > 0(V) be an effective irreducible representation of a connected 
compact Lie group G on a Euclidean space V. Assume that p is not polar. Then the following 
conditions are equivalent: 

(a) There is a representation r : H — )■ 0(W) of a compact Lie group with toric identity 
component such that the orbit spaces V/G, W/H are isometric. 

(b) There is a connected subgroup G of O(V) containing p(G) which acts polarly on V 
with cohomogeneity one less than the cohomogeneity of p. 

(c) The action of G leaves an isoparametric submanifold SofV invariant and acts with 
cohomogeneity one on S. 

Moreover, the representations satisfying any of the above conditions consist of three disjoint 
families as follows: 

(i) p is one of the non-polar irreducible representations of cohomogeneity three: 



G 


P 


Conditions 


SO(2) x Spin(9) 
U(2) x Sp(n) 
SU(2) x Sp(n) 


R 2 ® R R ib 

C 2 ® c C 2n 
S 2 {C 2 ) ® H c 2n 


n>2 
n>2 



(ii) The group G is the semisimple factor of an irreducible polar representation of Her- 
mitian type such that action of G is not orbit- equivalent to the polar representation: 



G 


P 


Conditions 


SU(n) 


S 2 C n 


n > 3 


SU(n) 


A 2 C n 


n = 2p > 6 


SU(n) x SU(n) 


C n ® c C n 


n > 3 


E 6 


c 27 





(iii) p is one of the two exceptions: SO(3) ® G 2 , SO(4) <S> Spin(7). 

A further motivation for Theorem 11.11 comes from one of the main results of |GLj . which 
gives another charaterization of the above class. Namely, if p : G — > 0(V), p' : G" — > 0(V') 
are two non-polar representations of a compact Lie groups with isometric orbit spaces V/G = 
V'/G' such that the restriction of p to the identity component G° is irreducible but that of 
p' to (G')° is reducible, then the restriction of p to G° satisfies the condition (a), for some 
representation r : H — >• 0(W). 

We are grateful to Wolfgang Ziller for pointing out the connection with the work of Kollross 
that considerably simplifies our classification. 

2. Non-classifying arguments 

Since principal orbits of polar representations are isoparametric submanifolds of Euclidean 
space [PT87j . (b) implies (c). Conversely, assume (c) holds. Then S is full and irre- 
ducible, thus either it is homogeneous or it has codimension two in V, due to the theorem 
of Throbergsson |Tho91] . In the former case, the maximal connected subgroup of O(V) 
preserving S acts polarly on V and has S as a principal orbit [PT87]. In the latter case, 
the cohomogeneity of G on V is 3. Such actions are listed in (i) above, and we can directly 
check that (b) holds in each case by finding a larger connected group acting as the isotropy 
representation of a symmetric space of rank two, namely, (SO(2) x SO(16),R 2 ®r R 16 ), 
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(U(2) x SU(2n), C 2 ® c C 2n ), (Sp(2) x Sp(n), C 4 ® H C 2n ), respectively. Hence (b) and (c) 
are equivalent. 

Next we prove that (a) implies (b). If r is as in (a), then the induced representation of 
H° is reducible, since H° is a torus. Using [GLl Theorem 1.7], we deduce that the action 



of H° on W can be identified with that of the maximal torus T k of SU(A; + 1) on C fc+1 , for 
some k > 1 (possibly after replacing H and r). Of course, H is contained in the normalizer 
N of H° in 0(W). Moreover, the connected component iV° is the maximal torus (of rank 
k + 1) of the unitary group of W, which acts polarly on W. 

Let X = V/G = W/H, let Y = X/(N/H) = W/N and denote by vr denote the composite 
map 

V -> X Y. 



This composite map 7r is a submetry (see |Lyt02 , iGWllj ). whose fibers are smooth equidis- 



tant submanifolds of V. Denote by Y reg the subset of principal iV/if-orbits in X, denote by 
V reg the set of G-regular points in V and put V = n^ 1 (Y reg ) R V reg . Then V is an open 
dense subset of V and it : V — > Y reg is a Riemannian submersion. Since the action of iV on 
W is polar, Y reg is flat, so O'Neill's formula implies that the 7r-horizontal distribution in V 
is integrable. Hence the regular fibers of / have flat normal bundles. Moreover, each regular 
fiber is equifocal (the focal points are given by the intersection of a horizontal geodesic with 
singular fibers). It follows that the fibers of it in V yield an isoparametric foliation J 7 by full 
irreducible submanifolds of V. 

We have codim J 7 = dimY = dimX — 1 = k + 1. In case k — 1, the cohomogeneity 
of p is 3 and the result follows as above, so we may assume k > 2. By the theorem of 
Thorbergsson |Tho91j . J 7 is homogeneous, and the maximal connected subgroup G C 0(V) 
which preserves the foliations acts transitively on the leaves. By definition, G is closed, acts 
polarly and contains G. By construction, V/G — Y. Hence we have (b). 

It was already remarked in |GLj that the representations in (i), (ii), (iii) all satisfy (a). 
More precisely, representations of cohomogeneity three have copolarity one ([GL, Exam- 
ple 1.9]; see also [Str94j). The representations listed in (iii) have copolarity 2 and 3 respec- 
tively [GL[ Theorem 1.11]. Thus, due to Theorem 1.5 in the same paper, the representations 
listed in (i) and (iii) satisfy (a). That the representations coming from the Hermitian sym- 
metric spaces and listed in (ii) satisfy (a) has been observed in [GL[ Example 1.10]; see 
also [GOT04] . 

We shall finish the proof of Theorem 11.11 in the next section by proving that the represen- 
tations satisfying (b) are listed in (i), (ii) and (iii) above. 



3. The classification 

Let G be as in (b). Consider the maximal connected subgroup K of 0(V) with the same 
orbits as G. It is known that the action of K on V is the isotropy representation of an 
irreducible symmetric space |BCO03[ Prop. 4.3.9], say, of compact type. Let (L,K) and 
M = L/K be the corresponding symmetric pair and symmetric space, respectively. The 
cohomogeneity c(K, V) is just the rank r = rk(M) of the symmetric space M. We have 
G C K and c(G, V) = c(K, V) + 1 = r + 1. We run over the cases of irreducible symmetric 
spaces of compact type [Hel78l IWo l84j . 
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3.1. Adjoint representations. These are associated to symmetric spaces of type II. For 
the adjoint representation of K on its Lie algebra £ we have that g, the Lie algebra of G, is a 
proper invariant subspace of t for the G-action, hence the action of G on t is not irreducible. 

3.2. List of symmetric spaces of type I. One says that the symmetric space M = L/K 
has maximal rank if rk(M) = rk(L). Such spaces are marked with * below. The space 
marked ** has maximal rank if and only if \p — q\ < 1. 



L/K 


dimpnsion 


rank 


SU(n)/SO(n)* 


ifn - l)(n + 2) 


n — 1 


SU(2n)/Sr>(n) 


(n — -\\(2n -4- 1 ^ 


Tl — 1 


S\J(v + a)/S(U(v) x Ufa)) 


2va 

^FH 


Ulin-fD Dr 


SOf v + a) /(SO(v) x SO (a)) * * 


FH 




SO(2n)/U(n) 


n{n — 1) 


[21 


SDfn)/Ufn)* 


n{n -\- 1) 


77. 


Sp(p + o)/(Sp(p) x Sp(o)) 


Ann 

F\t 


minip, o} 


E 6 /Sp(4)* 


42 


6 


E 6 /SU(6)SU(2) 


40 


4 


E 6 /Spin(10)U(l) 


32 


2 


E 6 /F 4 


26 


2 


E 7 /SU(8)* 


70 


7 


E 7 /Spin(12)SU(2) 


64 


4 


E 7 /E 6 U(1) 


54 


3 


E 8 /Spin(16)* 


128 


8 


E 8 /E 7 SU(2) 


112 


4 


F 4 /Sp(3)Sp(l)* 


28 


4 


F 4 /Spin(9) 


16 


1 


G 2 /SO(4)* 


8 
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3.3. Spaces of low rank. Here we deal with the case r < 4. We rely on the classification of 
irreducible representations of connected groups of cohomogeneity at most five (|Str96j and 
[UL~1 Theorem 1.11]). 

Recall that c(G, V) — r + 1. If r = 1, then c(G, V) — 2 and p is polar. 
If r = 2, then c(G, V) = 3 and p is listed in (i) [3tr96] . 

If r = 3, then c(G, V) = 4 and p is given in \GL\ Table 1]. The only cases in the table not 
listed in (ii) or (iii) are the first two, for which dimM = dimV^ < 8 and rk(M) = 3. But 
there are no symmetric spaces under these conditions. 

Finally, if r = 4 then c(G, V) = 5 and p is given in \GL\ Table 2]. The only cases in the 
table not listed in (ii) or (iii) are the first two and the last. In those cases the dimension 
of V is 8, 12 or 24. Going through the list of symmetric spaces of type I and rank 4, we 
see that dim(y) = dim(M) cannot be 8 or 12 and we are left with the 24-dimensional case 
M = SO(10)/(SO(6) x SO(4)). Thus we only have to exclude the case K = SO(6) x SO(4) 
and G = U(3) x Sp(2). However, U(3) x Sp(2) cannot be a subgroup of SO(6) x SO(4). 

Henceforth we shall assume r > 5. 

3.4. Spaces of maximal rank. In this case, the principal isotropy group K princ is finite, 
so also Gprmc is finite. Therefore G has codimension one in K, so it is a normal subgroup 
of K. Since K has a normal subgroup of codimension one, it has a normal (hence central) 
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subgroup of dimension 1. Thus K has a circle factor and the corresponding symmetric space 
M is Hermitian. Since the rank is maximal and at least 5, we deduce M = Sp(n)/U(n). 
In this case, there is a unique codimension one subgroup of K and we get one example 
(SU(n),,S 2 C n ) listed in (ii). 

3.5. Reformulation. There remain five classical families of symmetric spaces that we are 
going to analyse in the following. For easy reference, we list their isotropy representations 
together with the identity components of their principal isotropy groups. 



K 




-f^ • — (-^princ) 


Conditions 


Sp(n) 


[A 2 c 2n e c] R 


s P (i) n 


n > 6 


S(U(p) x XJ(q)) (p > q) 


C p ®c C 9 * 


S(U(p-g)xU(l) 9 ) 


p > g > 5 


SO(p) x SO(g) (p > q) 


R p <X>r R" 


SO(p-g) 


p > g + 2 > 7 


U(n) 


A 2 C n 


SU(2) 2 if n is even 


n > 10 




8X1(2)^11(1) if n is odd 




Sp(p) x Sp(g) (p > g) 


H p ® H H 9 * 


Sp(p- g )xSp(l) 9 


p > q > 5 



3.6. Relation with the work of Kollross. Since G acts on the Euclidean space V with 
cohomogeneity one less than K, the action of G on the principal orbits of K has cohomogene- 
ity one. Since those orbits are of type K/K princ , this means that G x K princ , and therefore 
G x H where H := K° linc , acts on the Lie group K with cohomogeneity one, where the first 
factor acts from the left and the second from the right. Such (and more general) cohomo- 
geneity one actions have been extensively studied by Kollross. He has classified such actions 
in the case of simple groups K, unfortunately, only up to orbit equivalence. More precisely, 
he has shown that for any such action, one finds larger connected groups G C G C K and 
H C H C K, such that the triple (G, K, H) is listed in |Kol02l Theorem B]. 

Using this theorem of Kollross it will be easy to finish the proof of our theorem. We only 
need to circumvent some minor difficulties related to the fact that our group K needs not 
be simple and that our pair (G, H) needs not be maximal. 

We will derive from |Kol02] the following: 

Lemma 3.1. Let F be R, C or H. Let K be respectively SO(n), SU(n), Sp(n) acting on 

V = F n , with n > 5. Let H be a subgroup of K . Assume that V decomposes into H -invariant 
subspaces as V — V\ © Vi © V3, with 1 < dimp(Vi) < dimp(T^) < dimp^). Assume that 
for a proper subgroup G C K the group G x H acts with cohomogeneity at most one on K . 
Then F ^ H. If F = C then n is even and G is contained in Sp(|). If K = SO(7) then 
G C G2. Finally, in all cases, V3 has either F-dimension or ¥ -codimension at most 3. 

Proof. Assume first that G acts reducibly on V, namely, leaves a proper subspace W of 

V invariant. Let Ky/ = {k G K\k ■ W = W}. Then O = K/Kw is the Grassmannian of 
dim(iy)-dimensional subspaces of V and, since G C Kw, the group H acts with cohomogene- 
ity at most one on O. However, this is impossible, since H preserves the relative positions 
to the Vi of any subspace W G O and this implies that O/H is at least two-dimensional. 

Thus G acts irreducibly on V. Consider now a maximal connected subgroup G of K 
containing G. Set H = (Ky. A )° . Then if is a maximal connected subgroup of K containing 
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H. Thus G x H acts on K with cohomogeneity at most one. Hence it must appear either 
in the list of Onishik |Oni62] . or in the list of Kollross. 

From the list of Onishchik, we only get K = SO (7) and G = G 2 . In the list of Kollross, 
we see the remaining statements. □ 

3.7. General consequences. In case K is a direct product K = K\ x K 2 , we denote by 
Gi and Hi, respectively, the projections of G and of H to Ki. Then 1 = d\m.(KjG x H) > 
dim(Kx/Gx x H x ) + dmv(K 2 /G 2 x H 2 ). 

In any case, observe now that for each one of the groups K appearing in Subsection I3.5[ 
the assumption that the rank is at least 5 and the structure of the principal isotropy group 
-fTprinc implies the following: for any simple non-abelian factor Ki of K, the triple (Gi, Ki, Hi) 
has the form as in Lemma [3.11 above . Moreover, the decomposition of the space F n into if;- 
irreducible subspaces has at least five summands. Next we proceed case by case. 

3.8. Symplectic cases. Assume that K = Sp(n) or K = Sp(p) x Sp(g). From Lemma [37X1 
we see that the projection of G to each factor Ki coincides with Ki. We deduce K = G, unless 
possibly K = Sp(g) x Sp(g) and G being the diagonal subgroup of K (up to conjugation), 
isomorphic to Sp(g). However, in the last case H = Sp(l) 9 and G x H cannot act on K 
with codimension 1 since dim(K) — dim(G) — dim(H) > 1. 

3.9. Real case. Assume that K = SO(p) x SO(g), with p > q + 2 > 7. The projection of H 
to SO(g) is trivial, thus the projection of G to SO(g) has codimension at most 1 in SO(q). 
Hence this projection coincides with SO(g). The projection of H to SO(p) fixes pointwise 
a 5-dimensional subspace. If p > 8 we find an if-invariant decomposition R p = V x © V 2 © ^3 
with dimension and codimension of V3 at least 4; and applying Lemma [3.11 we deduce that 
the projection G\ of G to SO(p) is SO(p). We are left with the case p = 7, q = 5. Then 
the projection Hi of H to SO(p) is SO (2) and, due to Lemma I3.1[ the group G\ must 
be a subgroup of the 14-dimensional group G2. Then G% x Hi cannot act on SO(7) with 
cohomogeneity 1. 

It follows that the projections of G to the simple factors of K coincide with the factors. 
Since by assumption p 7^ q, we get G = K. 

3.10. Complex case. Assume that K = U(n) with n > 10. Then, for the projection 
Hi of H to the semisimple part SU(n), the space C n decomposes into a sum of at least 
five ifi-invariant subspaces of complex dimension 2. From Lemma 13.11 we deduce that the 
projection G x of G to SU(n) coincides with SU(rt). Therefore G has codimension 1 in K. 
Hence G = SU(n) and we get one example (SU(n), A 2 C n ), where n is even, listed in (ii) 
(note that in case n is odd, the action of SU(n) is orbit-equivalent to that of U(n), which 
is polar |EH99j ). 

3.11. Complex Grassmanian. Assume that K = S(XJ(p) x U(g)) with p > q > 5. Apply- 
ing Lemma [3TT1 to the projection H 2 of H and G 2 of G to SU(g), we see that G 2 = SU(g), 
unless possibly q = 6 and G 2 C Sp(3). But in the latter case, H 2 = S(U(l) b ) and G 2 x H 2 
cannot act with cohomogeneity 1 on SU(6). 

Similarly, for the projection of G\ of G to SU(p) we deduce that G\ = SXJ(p), unless 
possibly p = 6 and G\ C Sp(3). In the latter case the projection H\ of H to SU(p) is equal 

to S(U(1) 6 ), which is again impossible by dimensional reasons. 
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Thus the projection of G to the simple factors of K is surjective. Now either the projection 
G of G to SU(p) x SU(g) is surjective, or p = q and the image G is the twisted diagonal 
subgroup {(g,a(g)) : g G SU(g)} where a is an automorphism of SU(g). The last case is 
again impossible by dimensional reasons. Thus G = SXJ(p) x SU(g). 

Therefore, G has codimension one in K. Hence G = SXJ(p) x SU(g) and we get one 
example (SU(p) x SU(p),C p ® C p ) listed in (ii) (note that in case p > q the action of 
SU(j?) x SU(g) is orbit-equivalent to that of S(U(jo) x U(g)), which is polar |EH99j ). 
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